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Introduction 


The scientific and engineering enterprise is itself increasingly global.... International boundaries have 
become considerably less important in structuring the conduct of research and development. 1 

Students in science and engineering (S&E) are preparing for careers in fields where international 
partnerships are increasingly important to advancing knowledge and discoveries. It has been over a 
decade since the National Science Board (NSB) highlighted the importance of international 
collaboration and called for increased government commitment to promoting international S&E 
research and education. 2 NSB called for the National Science Foundation (NSF) to take a leadership 
role in international S&E research and education activities promoting "...increased participation in 
international S&E activities by younger U.S. scientists and engineers from diverse backgrounds, 
especially those in the early stage of their careers, in order to develop an internationally competitive 
and globally-engaged S&E workforce." 3 

NSF's East Asia and Pacific Summer Institutes (EAPSI) program, which provides international 
fellowships to U.S. graduate students in S&E, is one program that helps to further NSF's 
commitment to support the active engagement of early career S&E researchers in international 
collaborations. 

NSF contracted with Abt Associates to conduct an evaluation of the EAPSI program to investigate 
whether it was meeting its goal of providing U.S. graduate students with international experiences 
that will enable future collaboration with foreign peers. This report summarizes the findings from 
that evaluation, which was designed to investigate the characteristics of EAPSI applicants and their 
motivations for participation, the host researchers' motivations for participating in the program, 
participants' experiences in and perceptions of the program, and outcomes of the program. The full 
evaluation report is available online at http://www.abtassociates.com/Reports/2012/Evaluation-of- 
the-East-Asia-and-Pacific-Summer-lns.aspx. 4 


1 National Science Board. 2001. Toward a More Effective Role for the U.S. Government in International 
Science and Engineering. NSB-01-187. National Science Foundation, Arlington, VA., p. 12. 

2 National Science Board. 2001. 

3 National Science Board. 2000. Toward a More Effective NSF Role in International Science and Engineering, 
Interim Report. NSB-00-217. National Science Foundation, Arlington, VA., p. 10 

4 Martinez, A., Neishi, K. Parsad, A. , Whittaker, K., & Epstein, C. 2012. Evaluation of the East Asia and 
Pacific Summer Institutes Program: Final Report. Abt Associates Inc., Cambridge, MA. 
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Summary of Findings 


The evaluation found evidence that the EAPSI program is meeting its goals to: 


• Introduce U.S. graduate students to East Asia and Pacific science and engineering in the 
context of a research setting; and 

• Help students initiate scientific relationships that will better enable future collaboration 
with foreign counterparts. 


The evaluation also found evidence of broader benefits, finding that the program helps to orient 
fellows to the host's society, culture, and language, and provides benefits that extend beyond the 
EASPI fellows. 


Key Findings 


To investigate the progress of the program toward its goals, the evaluation paid particular attention 
to the opportunities for international research collaborations, the experiences that develop research 
capacity and global perspectives, and the relationships between U.S. and foreign researchers. 

1. What are the program experiences of program participants and managers? 

2. Does the extent to which former fellows engage in international collaborations differ from 
that of unfunded applicants? 

3. Do fellows' post-award career activities and job characteristics differ from those of 
unfunded applicants? 

4. What are the perceived outcomes of program participation? 

5. Do the outcomes of program participation extend beyond the direct participants? 

Introduce Students to East Asia and Pacific Science 

Research experiences that included collaborations with 
foreign scientists at their host sites were central to the 
fellows' EAPSI program experiences. 

• Overall, fellows were satisfied with their EAPSI 
research activities and interactions with their host. 

All EAPSI fellows would recommend the fellowship 
to another graduate student seeking an 
international experience, 92 percent of advisors 
would or have recommended the program to other 
graduate students, and 79 percent of hosts would 
or have recommended hosting an EAPSI fellow to 
others. 


and Engineering Research 


I believe the EAPSI experience was 
invaluable to my career. It gave me 
a unique opportunity I may not 
otherwise have had. It has shown 
me that international collaboration 
can be very fruitful, especially in 
combining research expertise from 
my home and host labs. (EAPSI 
fellow) 
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• EAPSI offers graduate students an opportunity to conduct research in foreign locations and to 
establish relationships that they may build on subsequently in their careers. A majority of 
fellows reported that they were very satisfied with their host's expertise (81 percent) and the 
match between their research interests and the host's (63 percent). 

• A minority of fellows noted difficulties related to their research activities and host interactions. 
Among the difficulties cited were not enough guidance from the host (17 percent), and being 
assigned a role that was less than merited by their skills or knowledge (5 percent). Less than 10 
percent of hosts reported specific challenges regarding collaborating on research with their 
fellow. 

Initiate Relationships That Enable Future International Collaborations 

A key goal of the EAPSI program is to help students initiate scientific relationships that will better 

enable future collaboration with foreign scientists. Although fellows and unfunded applicants 

similarly work with individuals in other countries, the evaluation provides evidence that the EAPSI 

experiences lead to more productive international research collaborations. 

• Although EAPSI forges new relationships for the 
fellows, about half of the fellowships build on 
some existing relationship. Only 15 percent of 
hosts reported that they knew their prospective 
fellow prior to him/her applying. Almost half (45 
percent) of the hosts, however, knew the 
graduate advisor of the prospective fellow. 

• Fellows had the opportunity to work with a range 
of individuals at the host institution. Hosts most 
frequently reported that their fellow worked 
with the host (80 percent), graduate students in 
the host's research group (66 percent), and other 
scientists in the host's research group (45 
percent). 

• A majority of the fellows indicated that their relationship with the host has continued beyond 
the fellowship period, through subsequent research collaborations with the host (20 percent) 
and/or additional communications (60 percent). Hosts had similar responses to the fellows, as 
29 percent stated that they had further collaborated with the fellow and 48 percent had 
communications with the fellow post-fellowship. 

• Unfunded applicants continued to pursue other avenues for their international experiences. 
EAPSI fellows and unfunded applicants with PhDs held a similar number of international 
postdoctoral fellowships (0.32 versus 0.28 fellowships, respectively), yet fellows were less likely 
than unfunded applicants to have been employed outside the U.S. (13 versus 39 percent, 
respectively). 

• Forty percent of fellows and 35 percent of unfunded applicants reported working with 
individuals located in other countries. Among those who work with a collaborator in a foreign 


The EAPSI program allowed the 
establishment of an on-going 
collaboration between my lab and 
that of the U.S. graduate mentor. A 
peer-reviewed manuscript is under 
review at the moment from the work 
completed by the fellow (and the 
subsequent work by new members 
of my lab). I am organizing a visit to 
the U.S. lab by one of my current 
graduate students as part of this 
collaboration. (EAPSI host) 
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country, unfunded applicants were statistically more likely to report that their collaboration 
included joint publications or jointly developed products (82 versus 92 percent). 

• Compared to unfunded applicants, fellows reported a higher number of publications co- 
authored with a foreign collaborator (2.2 versus 1.6 publications) and a larger proportion of 
publications that include a foreign co-author (22 versus 17 percent), suggesting that fellows 
have more productive collaborations with foreign counterparts than do unfunded applicants. 

• Benchmarking analyses comparing career outcomes of EAPSI fellows to national estimates for 
similar S&E PhD graduates found that EAPSI fellows were more likely to be working with 
individuals in other countries (80 percent of fellows holding a PhD versus 30 percent of the 
national sample). 

Extend Benefits Beyond EAPSI Fellows 

The experiences extend beyond the specific participants and seed additional international S&E 

research activities. As such, EAPSI contributes to NSF's efforts to promote international science and 

engineering among early career scientists and engineers. 

• As reported by host site representatives, host countries become involved in EAPSI to increase 
research collaborations with the U.S., to establish a research network and ongoing relationships 
between scientists, and to provide younger researchers from the U.S. with an opportunity to 
understand the R&D activities in their countries. 

• About one-third of fellows reported that other researchers from their home institution became 
involved in the research project during the EAPSI fellowship. 

• EAPSI fellows continue to promote international collaborations after they return to the U.S. 
More than half of the fellows engaged in activities where they extended the benefits of their 
EAPSI participation with others, particularly by sharing resources they collected or tools 
developed during their fellowship (59 percent) and teaching others about the research methods 
they learned during that time (56 percent). 


• Fellows were significantly more likely than 
unfunded applicants to engage in a series of 
activities to foster international 
collaborations among others (40 percent 
versus 30 percent, respectively). 




• Host site representatives identified several 
benefits of the program for those involved- 
fellows, other students in the host lab or 
institution, host researchers, and the host 
institutions and countries. Respondents 
commonly cited increased opportunities for 
collaboration and networking as benefits of 
participation. 



experimental system and a new idea into 
my group. The experience working on a 
new experimental system was quite 
stimulating for me. She worked 
efficiently, quickly setting up 
experimental conditions to obtain 
reliable results, and made successful 
accomplishments in the short period of 
her stay. (EAPSI host) 



Abt Associates 


5 


Emerging International Researchers: Key Findings from the Evaluation of NSP’s EAPSI Program 


• Host researchers most commonly reported that they agreed to serve as a host in the program 
because of a shared interest in the prospective fellow's research project (64 percent), an 
interest in creating an international environment in their research group (59 percent), and to 
establish or maintain collaboration with a U.S. researcher (57 percent). 

• As a result of the EAPSI experience, most hosts identified some benefit; only 11 percent 
perceived no personal benefits from their participation. The most common benefits selected by 
respondents included enhanced interest in collaborating with U.S. researchers (31 percent), 
established or renewed collaborations with other U.S. researchers (30 percent), and published 
papers based on the research conducted (25 percent). 


Additional Findings 


The evaluation was also designed to address the following questions: 

1. What are the characteristics of people who apply for and participate in the EAPSI program? 

2. What motivates individuals to apply for and participate in the program, and what are 
individuals' experiences during the application process? 

The EAPSI applicants represent graduate students with varied backgrounds and motivations for 
participating in the program. 

• Applicants most commonly applied to EAPSI to enhance their skills or knowledge as a researcher 
(82 percent). 

• The majority of EAPSI applicants did not have extensive academic or travel experiences outside 
the U.S. at the time of application. Thus, through the research opportunities, EAPSI introduced 
fellows to international experiences. Only 36 percent of applicants participated in study abroad 
programs as undergraduates or graduate students, and 18 percent had attended elementary or 
secondary school outside the U.S. Around one-third had lived outside the U.S. for six months or 
longer (35 percent) or visited or lived in one of the seven EAPSI host locations for a month or 
longer (30 percent). Less than a quarter of applicants (22 percent) reported that they had 
attended a research conference outside the U.S. 

• A majority of hosts agreed that that their research interests were well-matched with those of 
their fellow (82 percent), and that they and the fellow shared similar goals and expectations for 
the EAPSI experience (60 percent). 

Applicants are also drawn to EAPSI in part because it provides an opportunity to learn about the 
society, culture, and language of host sites. Nearly all fellows reported participating in cultural and 
leisure activities while abroad, and a larger majority reported becoming comfortable with the 
traditions and culture of their host site. 

• Applicants commonly reported applying to EAPSI in part to learn about the culture, history, and 
geography of another country (77 percent), and to conduct research with a specific person or at 
a specific institution (70 percent). 
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• Nearly all fellows participated in cultural and leisure activities in their host site which included 
sightseeing (97 percent), exploring the landscape or geography (91 percent), and visiting 
museums (89 percent). 

• Among fellows placed in primarily non-English speaking host sites— Japan, Korea, Taiwan, and 
China— 67 percent reported some type of language training in preparation for their 
international fellowship experience. This included self-guided study (47 percent), a formal 
language training course (23 percent), and working with a conversational partner or tutor (9 
percent). 

• Communication or language issues were the most frequently cited difficulties that fellows 
encountered during their fellowship; however, this varied significantly by host site where a 
greater number of fellows in primarily non-English speaking host sites experienced these 
difficulties (28 to 44 percent within these sites) than fellows in primarily English-speaking host 
sites— New Zealand, Australia, and Singapore (0 to 10 percent within these sites). 


Evaluation Approach 


The evaluation was designed to assess how the EAPSI program contributes to the engagement of 
early career S&E researchers in international research collaborations. The study incorporated both 
extant and primary data sources. These included surveys administered to program applicants (both 
those who received EAPSI fellowships and those who did not), EAPSI foreign hosts, and EAPSI 
graduate advisors, as well as telephone interviews with EAPSI agency officials and managers at 
foreign counterpart agencies. The study included both descriptive and comparative analyses, 
including a rigorous quasi-experimental design to answer the questions about program impacts. 

Propensity score analysis (PSA) was used to construct groups of awardees and non-awardees that 
were statistically similar across a number of pre-existing characteristics, in order to compare the 
outcomes of fellows to those of unfunded applicants, using pre-award characteristics of applicants 
to mitigate the potential threat of selection bias. To situate the EAPSI program participants' and 
applicants' outcomes within the national S&E context, a secondary set of comparative analyses was 
conducted using data from the Survey of Doctoral Recipients (SDR) and the National Survey of 
Recent College Graduates (NSRCG). The evaluation also used descriptive analyses to explore the pre- 
award international research experiences and other characteristics of EAPSI applicants and host 
scientists; to understand what motivated STEM graduate students to apply for an EAPSI fellowship 
and what led EAPSI host scientists to host a fellow to describe the perceptions and role of EAPSI 
advisors; and to describe the experiences of EAPSI participants (i.e., fellows and hosts), both during 
and after the period of the fellowship. 


Summary 


The evaluation of EAPSI examined the characteristics and motivations of EAPSI participants, the 
opportunities for professional and personal growth provided through EAPSI, and the professional 
outcomes related to the program. Findings from this study provide evidence that EAPSI is 
contributing to NSF's efforts to help develop an internationally competitive and globally engaged 
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S&E workforce. Specifically, the evaluation found evidence that EAPSI is meeting its goals: to 
introduce U.S. graduate students to East Asia and Pacific S&E in the context of a research setting and 
help students initiate scientific relationships that will better enable future collaboration with foreign 
counterparts. The evaluation also found evidence of ancillary benefits of the program, which include 
orienting fellows to the host's society, culture, and language, and providing opportunities whose 
benefits extend beyond the EASPI fellows. Below, the program's progress toward its stated goals as 
well as its broader influences are summarized. 


Introduce Students to East Asia and Pacific Science and Engineering 
Research 


Research experiences that included collaborations with foreign scientists at their host sites were 
central to the fellows' EAPSI program experiences. The majority of EAPSI applicants did not have 
extensive academic or travel experiences outside the U.S. at the time of application. Thus, through 
the research opportunities, EAPSI introduced fellows to international experiences. 

Reflecting the globalization of science, both home departments and host departments were 
generally supportive of international research. Both fellows and unfunded applicants reported being 
attracted to the program because it offered them an opportunity to enhance their skills and 
knowledge as a researcher. The fellowships provided opportunities for fellows to engage in research 
activities independently as well as collaboratively, and fellows reported being satisfied with the 
various aspects of their research and their interactions with their hosts. Overwhelmingly, EAPSI 
fellows would recommend the fellowship to other graduate students seeking an international 
experience. 

Graduate students apply to EAPSI because they are interested in pursuing an international research 
experience. Several findings suggest that unfunded applicants continue to pursue other avenues for 
their international experiences. Among EAPSI applicants who held a PhD, EAPSI fellows and 
unfunded applicants held a similar number of international postdoctoral fellowships, and EAPSI 
fellows were less likely than unfunded EAPSI applicants to have been employed outside the U.S. 


Initiate Relationships That Enable Future International Collaborations 


A key goal of the EAPSI program is to help students initiate scientific relationships that will better 
enable future collaboration with foreign scientists. The evaluation provided evidence that the 
fellows initiate scientific collaborations during their fellowships and that fellows and advisors 
perceived the fellowships as expanding the opportunities available to fellows. Over half of the 
fellows indicated that their participation led to valuable connections to researchers outside the U.S., 
and the majority of host and fellows reported follow-up communications or collaborations after the 
fellowship had ended. 

Fellows also continue productive working relationships with individuals outside the U.S. Although 
unfunded applicants who work with a collaborator in a foreign country were more likely than 
fellows to characterize their collaborations as ones that involved joint publications or jointly 
developed products, for actual products, fellows reported a higher number of publications and a 
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higher proportion of their publications that were co-authored with a foreign collaborator, providing 
evidence that the EAPSI experiences lead to more productive international research collaborations. 


Orient Fellows to Host Society, Culture, and Language 


Applicants are also drawn to EAPSI in part because it provides an opportunity to learn about the 
society, culture, and language of host sites. Nearly all fellows reported participating in cultural and 
leisure activities while abroad, and a larger majority reported becoming comfortable with the 
traditions and culture of their host site. 

Many fellows who went to locations where English is not a major language reported engaging in 
language training. Nonetheless, some fellows still reported challenges related to communication 
and language, although most hosts did not report challenges related to the lack of familiarity. 


Extend Benefits Beyond EAPSI Fellows 


The EAPSI fellowship experience and benefits also extend to researchers in the U.S. For example, 
during the EAPSI fellowships, about one-third of fellows reported that other researchers from their 
home institution became involved in the research project. They continue to promote international 
collaborations after they return to the U.S.; specifically, the evaluation found that fellows were more 
likely than unfunded applicants to engage in a series of activities to foster international 
collaborations among others. 

Among the reasons that foreign locations gave for becoming involved in EAPSI were to increase 
research collaborations with the U.S. and to establish a research network and ongoing relationships 
among scientists. Hosts, also, were drawn to EAPSI both by their interest in the research projects of 
the fellows and by the benefits they feel might accrue to their research groups in addition to 
themselves. Some pointed to specific benefits such as increased interest in collaborating with U.S. 
researchers and/or new or renewed collaborations with U.S. researchers. 


Conclusion 


The evaluation demonstrated that participants derive benefits from the program, on both an 
individual and a collective basis. EASPI fellows and hosts recommend the EAPSI program to their 
colleagues, and EAPSI U.S. advisors would also encourage additional students to participate. EAPSI 
offers graduate students an opportunity to conduct research in foreign locations and to establish 
relationships that they may build on subsequently in their careers. The experiences also extend 
beyond the specific participants and seed additional international S&E research activities. As such, 
EAPSI contributes to NSF's efforts to promote international S&E among early career scientists and 
engineers. 
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